Practical IDS alert correlation in the face of dynamic threats

A significant challenge in applying IDS alert correlation in today’s dynamic threat environment is the labor and expertise needed in constructing the correlation model, or the knowledge base, for the correlation process. New IDS signatures capturing emerging threats are generated on a daily basis, and the attack scenarios each captured activity may be involved in are also multitude. Thus it becomes hard to build and maintain IDS alert correlation models based on a set of known scenarios. Learning IDS correlation models face the same challenge caused by the dynamism of cyber threats, compounded by the inherent difficulty in applying learning algorithms in an adversarial environment. We propose a new method for conducting alert correlation based on a simple and direct semantic model for IDS alerts. The correlation model is separate from the semantic model and can be constructed on various granularities. The semantic model only maps an alert to its potential meanings, without any reference to what types of attack scenarios the activity may be involved in. We show that such a correlation model can effectively capture attack scenarios from data sets that are not used at all in the model construction process, illustrating the power of such correlation methods in detecting novel, new attack scenarios. We rigorously evaluate our prototype on a number of publicly available data sets and a production system, and the result shows that our correlation engine can correctly capture almost all the attack scenarios in the data sets.